Modernising a thousand of Phaser examples to ES6

2022-07-22

When I started working with the Phaser examples repository, there was roughly 1.700~ examples, from which 1.298 needed some serious JavaScript renewals.

Doing it manually would take a lot of commits, so in trying to avoid such inconvenience, I wanted to automate it. While 1.298 is not a very large number, every second shaved with automation from potential 1.200+ changes needed would save 20 minutes of mundane work.

(xkcd: Automation)

Thankfully, there are plenty of tools in the JavaScript ecosystem that handles code transformations so I had high hopes for truly automating it. For this task, I had in mind two:

  • Lebab, which transforms ES5 code into ES6.
  • jscodeshift, a tool based on recast, which manipulates the JavaScript syntax tree.

The rough idea was to run a set of transformations with lebab and make the specific changes the Phaser examples needed with jscodeshift.

The last piece was making sure the examples that were working before making any changes would work afterwards. With puppeteer and some code borrowed from around the repository, I setup an automatic error collection running with the examples. If the transformation would break the example's JavaScript, there was a way to find out.

In the end, the tooling had this shape:

for each example file:
  - filter out the ones with ES6 class (that means they are fresh)
  - browser test the example, log any errors
  - transform code
  - browser test the converted code, log
  - log results

Work to be done

Fortunately, for automation, Phaser examples have a pretty stable shape. On average, ES5 samples in the repository had two functions declared, some configuration object, and sometimes a couple of global variables.

var config = {
  // ...
  scenes: {
    preload: preload,
    create: create,
  },
};

var game = new Phaser.Game(config);

var someVariable;

function preload() {
  /* ... */
}
function create() {
  /* ... */
}

The first step and early goal was to get these "normalised" examples to use ES6 classes.

class Example extends Phaser.Scene {
  someVariable;

  preload() {
    /* ... */
  }
  create() {
    /* ... */
  }
}

const config = {
  // ...
  scenes: Example,
};

const game = new Phaser.Game(config);

A couple of things had to happen:

  • var → const to be handled by lebab.
  • function → class to be built with a codemod using jscodeshift.
  • globals → class fields also with codemods.

And once I had an early version working with only those three transforms, the early success rate of conversion was flying around 80%. Yet another win of Pareto principle. At this point, the resulting code was not totally functional examples, only valid JavaScript, but nevertheless these were encouraging numbers.

Writing those three transforms

For lebab it was using a configuration and calling the API. The two other were jscodeshifts codemods.

As I mentioned above, jscodeshift manipulates the JavaScript syntax tree and can give back new code with the transformations. Yes, no Regexes are involved and there's no need to invoke THE PONY.

A very simple transformation looks like this:

const j = jscodeshift;
const rootSource = j(file.source);

j.find(j.Identifier).forEach((path) => {
  // do something with path
  j(path).replaceWith((p) =>
    j.identifier(p.node.name.split("").reverse().join(""))
  );
});

return rootSource.toSource();

Here is what going on the code above:

  • Line 2: Create the AST from our source code.
  • Line 4: Find all the identifiers in the code.
  • Line 7: Replace those identifiers nodes on the AST with the same identifier, but reversed.
  • Line 13: Generate the JavaScript code back, which will contain the transformations.

The core of writing these transforms then is knowing which nodes to move and work with. How one knows all this sorcery? Don't. Just use the source.

The AST explorer is an essential tool and allows one to explore the source code of a surprisingly large amount of languages. My workflow was basically pasting example code into AST explorer and figuring out where I needed to make the incision.

Another indispensable resource to understand better the APIs of jscodeshift was benjamn/ast-types. The documentation describes all the elements of traversing the AST, what is what when you run the searches and which APIs are available to help you.

ASTs are fun

Any search online will give you plenty to start coding codemods. What I didn't find was anything about the trick cases, or proper usage. Below are some of the situations I got myself into.

Also, you can find and reference the full codemod I wrote for the transformation of the code examples in my fork of phaser3-examples.

Phaser.Class

Phaser.Class were a fun one to make (code here) and showed how interesting and powerful ASTs are.

var SceneA = new Phaser.Class({
  Extends: Phaser.Scene,

  initialize: function SceneA() {
    Phaser.Scene.call(this, { key: "sceneA" });
  },

  preload: function () {
    /* ... */
  },
  create: function () {
    /* ... */
  },
});

The trick detail was to move the scene configuration from the Phaser.Scene.call into the super call inside the class constructor.

class SceneA extends Phaser.Scene {
  constructor() {
    super({ key: "sceneA" });
  }

  preload() {
    /* ... */
  }
  create() {
    /* ... */
  }
}

To use or not to use paths

Overall during my initial spike, I was confused to whether should I be wrapping everything into jscodeshift's or not, if I needed to .get() or not and so on.

Don't do like me and as I mentioned earlier, read the docs benjamn/ast-types. I missed this mention from jscodeshift documentation and it hurt, so much that I'm double-mentioning this page here. It could have helped me understand the difference between Node and NodePath, for example.

Global declarations

This was a very minor thing, but it took me some tries to figure out so I thought I should just leave it here as no stackoverflow search saved me here.

// Finding global variables
root
  .find(j.VariableDeclaration)
  .filter((path) => j(path).closestScope().isOfType(j.Program));

And then some better version later, when I was wiser ;P

root
  .find(j.FunctionDeclaration)
  .filter((path) => path.parentPath.scope.isGlobal);

Identifiers are everywhere

On early code, trying to not make a lot of assumptions and trusting Phaser examples, I moved all the global variables into the class as class parameters (properties in the API).

exampleClass.body.body.unshift(
  j.classProperty(j.identifier(memberName), memberValue)
);

And in order then to make these valid, I need to transform the usages of those variables from variable to this.variable.

root
  .find(j.Identifier, {
    name: memberName,
  })
  .forEach((p) => {
    j(p).replaceWith(
      j.memberExpression(j.thisExpression(), j.identifier(memberName))
    );
  });

But then I run into functions reusing names of global variables as their parameter

var color;

// ...

function paintSquare(ctx, color) {
  // The parameter color overrides the global color
  ctx.setFill(color);

  // ...
}

Which made the codemod generate a funny-looking invalid JavaScript:

class Example extends Phaser.Scene {
  color;

  // ...

  paintSquare(ctx, this.color) {
    // Erm...
    ctx.setFill(this.color);

    // ...
  }
}

The solution was to filter out any function parameters and the usage inside the functions.

root.find(j.Identifier, {
  name: memberName,
})
  .filter((path) => {
    // Creates an array to which the identifier shouldn't be updated
    if (path.parentPath.name === "params") {
      ignoreScopes.push(path.parentPath.scope.node);
    }

    if (
      ignoreScopes.indexOf(path.parentPath.scope.node) > -1
    ) {
      return false;
    }
    // ...

But it was not enough. Later I also needed to filter more things like property accessing in objects (this.rectagle.color that was ending like this.rectangle.this.color), and left side assignments (let color becoming let this.color). I ended up with a test bed for filtering out which transformations needed to be done astexplorer.net gist.

And it's not all

There are about 150 examples left to be transformed to ES6, roughly 10%. And those are the ones that will require manual review.

For example, var context scoping tripped the automatic error detection on plenty of examples, as lebab couldn't make the transformation safely. Others, may have some extra configuration on the game object, and yet others are simply some dwitter code-golfs.